Methods and system for efficient processing of generic geometric correction engine

ABSTRACT

An apparatus and method for geometrically correcting a distorted input frame and generating an undistorted output frame. The apparatus includes an external memory block that stores the input frame, a counter block to compute output coordinates of the output frame for a region based on a block size of the region, a back mapping block to generate input coordinates corresponding to each of the output coordinates, a bounding module to compute input blocks corresponding to each of the input coordinates, a buffer module to fetch data corresponding to each of the input blocks, an interpolation module to interpolate data from the buffer module and a display module that receives the interpolated data for each of the regions and stitch an output image. The method includes determining the size of the output block based on a magnification data.

BACKGROUND OF THE INVENTION Technical Field

The present invention relates to geometric correction engines and morespecifically to correcting distorted images captured by lenses andgeometrically correcting to generate an undistorted image.

Description of Related Art

Lens geometric distortion issues in a camera system are common. Thedistortion can be common optical distortions, such as barrel distortion,pin-cushion distortion, or fisheye distortion. A correction therefore isneeded to correct these types of distortions in hardware in combinationwith software or firmware. Affine transformation support is added tosupport the image and video stabilization application, where multipleimages of the same scene need to be aligned. Perspective warp is anextension to affine transform and offers additional capabilities.Perspective transformations can align two images that are captured fromdifferent camera viewpoints or locations. This can also be used torectify the left and right images of a stereo camera to reduce thecomplexity of a disparity computation. Perspective transformations canalso generate new viewpoints.

Most digital cameras suffer from some degree of non-linear geometricdistortion. A spatial transformation is required to correct thedistortion. In automotive and security applications, cameras use wideangle lenses, including fisheye lenses to provide 180+° field of view.To visually present the scene to the user in an easy-to-consumerepresentation, these distortions need to be corrected.

FIG. 1 (0100) generally illustrates a distorted image (0101)geometrically corrected with a geometric correction engine (0102) anddisplayed as an undistorted image (0103). One technique to correct adistorted image is forward mapping which gives coordinates of theoriginal, undistorted image as a function of coordinates of thedistorted image. Forward mapping maps the input pixels (distorted image)onto the output space (undistorted image). Using forward mapping, everyoutput pixel may not correspond to a valid input location, so outputpixels can be empty or contain many nearby mapped pixels requiringcomplex interpolation algorithms. Therefore, forward mapping is notsuitable for correcting the distortion.

Another technique is back mapping which gives coordinates of thedistorted image as a function of coordinates of the undistorted outputimage. Correction involves back-mapping each output pixel to a locationin the source distorted image, and thus the corrected image is fullypopulated. As the distorted pixel locations mostly fall onto fractionalcoordinates, correction involves interpolation among the nearestavailable pixels.

FIG. 2 (0200) generally illustrates an output image (0202) back mappedto an input image (0201). The input image (0201) is blended with fetchblocks such as blocks (0203, 0204, 0223, 0233). The fetch blocks containpixel information of input blocks. For any distortion, the user maydetermine the overlap between the distorted space and the input imagecoordinates. Valid image data is produced only for those points wherethe distorted space lies completely within the valid input image.

The output image is divided into output blocks containing pixelinformation of the output image, for example blocks (0213, 0214) containinformation of parts of the mat in the output image. The output blockshave a fixed size with a constant width and a constant height. The widthof the output block represents the number of pixels in the blockarranged in a row. The height of the output block represents the numberof lines in the block. In current back mapping system, each of theoutput blocks is back mapped into input blocks after a perspectivetransform and pixel interpolation. For example, output block (0214) isback mapped to input block (0204) representing a portion of the whitecolored border of the mat in the image (0202). Similarly, output block(0213) is back mapped to input sub blocks (0203, 0223, 0233)representing data corresponding to top right corner data from the centerwhite square of the mat. With a fixed output block size, it is clearthat certain output blocks require less information compared to otheroutput blocks. In order to back map after perspective transform, outputblock (0214) required data from one input block (0204), while outputblock (0213) required data from 3 input sub blocks (0203, 0223, 0233).The input blocks required for an output block may be together or spreadapart in the input image. The input fetch blocks are stored off chipgenerally in a DDR memory. When the input blocks are together, the DDRbus may accommodate fetching the required input blocks in one cycle.However, if the input blocks are spread apart, multiple cycles may berequired to fetch the input block due to addressing constraints and buswidth. Latency difference for fetching is different for different inputblocks resulting in performance loss and poor utilization of bandwidth.Therefore, with a fixed output block size, some output blocks requireless information to be fetched versus some others. The ratio of the sizeof the input block size to the size of the output blocks required forthe output block may be defined as a magnification factor or scalingfactor. The size of the output block size is determined by the largestmagnification factor. In other words, the smaller the output block sizethe larger the amount of data required to be fetched into an internalmemory. The internal memory is fixed in size and the data required to befetched for a high magnification factor output block may require manycycles to fetch the data resulting in a high bandwidth use and loss ofperformance. Therefore, there is a need to program the size of theoutput block size based on the magnification factor so that thebandwidth and cycles for fetching the input block data into an internalmemory is minimized resulting in an enhanced performance.

FIG. 3 (0300) illustrates a prior art input image captured by multiplecameras and stitched into an output image. An output image (0310) iscomposed and stitched from multiple input images (0301, 0302, 0303,0304) captured from different angles by different cameras. The camerasmay be dash mounted on the front left, front right, rear left, and rearright corners of an automobile. In FIG. 3, the cameras are mounted onfront, back, left and right. The images thus captured are stitched intoan output image. However, the multiple images captured from differentviewpoints contain redundant information. For example, the region (0314)captured by a central camera is also captured in the region (0313) by aright camera and captured in the region (0315) by a left camera. Theredundant regions are double processed for stitching and processing thesame image. There is considerable loss of performance due to cyclesspent in fetching and processing the redundant blocks. Therefore, thereis a need to identify and skip the processing of the redundant blocks.

FIG. 4 (0400) illustrates a prior art geometric correction block diagramof an input frame buffer to generate an output frame buffer through backmapping and mesh data. An output frame buffer (0402) is mapped to aninput frame buffer (0401) after each of the output blocks (0403) in theoutput frame buffer (0402) undergoes a perspective transformation andmesh warping through a geometric correction engine (0404) to generate aninput block (0405). The output of the perspective warp may be anintermediate block (0406) corresponding to the four corners of theoutput block (0403). A mesh table (0407) that maintains the informationcorresponding to the output block or the intermediate block may furtherbe used to map the intermediate block to the four corners of an inputblock (0405). Prior art output block has a fixed size and therefore allthe pixel data mapped to the input blocks which can contain unused dataneed to be fetched together. Therefore, there is a need for a method tocustomize the output block size based on the scaling factor.

SUMMARY OF THE INVENTION

The present invention is an apparatus and method for geometricallycorrecting a distorted input frame and generating an undistorted outputframe. The apparatus includes an external memory block that stores theinput frame, a counter block to compute output coordinates of the outputframe for a region based on a block size of the region, a back mappingblock to generate input coordinates corresponding to each of the outputcoordinates, a bounding module to compute input blocks corresponding toeach of the input coordinates, a buffer module to fetch and store datacorresponding to each of the input blocks, an interpolation module tointerpolate data from the buffer module, and a display module thatreceives the interpolated data for each of the regions and stitch anoutput image. The method includes determining the size of the outputblock based on magnification data and identifying redundant regions inthe output image.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the invention are setforth in the appended claims. The invention itself, however, as well asa preferred mode of use, further objectives and advantages thereof, willbe best understood by reference to the following detailed description ofillustrative embodiments when read in conjunction with the accompanyingdrawings, wherein:

FIG. 1 illustrates a prior art distorted input frame geometricallyconverted to an output frame.

FIG. 2 illustrates a prior art output frame divided into blockscorresponding to blocks in a captured input frame.

FIG. 3 illustrates a prior art input image captured by multiple camerasand stitched into an output image.

FIG. 4 illustrates a prior art geometric correction of an input framebuffer to generate an output frame buffer through back mapping and meshdata.

FIG. 5 depicts an exemplary histogram of scaling data of output blocksof FIG. 2, and a spatial plot of the scaling data of the output blocksin an output frame based on the scaling thresholds according to oneaspect of the present invention.

FIG. 6 is a depiction of an exemplary output frame divided into regionsand each of the regions divided into blocks with different sizes basedon the scaling data according to one aspect of the present invention.

FIG. 7 illustrates an exemplary image processing system according to oneaspect of the present invention.

FIG. 8 illustrates an exemplary diagram of a spatially adaptive slicingapparatus according to one aspect of the present invention.

FIG. 9 illustrates an exemplary diagram of a perspective coordinatedetermination apparatus according to one aspect of the presentinvention.

FIG. 10 illustrates an exemplary block diagram of back mapping outputcoordinates in an output block to input coordinates in an input blockand interpolation block according to one aspect of the presentinvention.

FIG. 11 is an exemplary flowchart of a method for correcting an inputframe and generating an output frame with a spatial slicing apparatusaccording to one aspect of the present invention.

FIG. 12 illustrates an exemplary diagram of an output frame withredundant blocks identified in regions according to one aspect of thepresent invention.

FIG. 13 illustrates an exemplary diagram of an output frame withredundant blocks skipped in regions in multi frame processing accordingto one aspect of the present invention.

FIG. 14 is an exemplary flowchart of a method correcting an input frameand skipping redundant blocks in generating an output frame and decidingregions for best approximation to avoid unused portion of the imageaccording to one aspect of the present invention.

FIG. 15 illustrates an output block in an output frame dynamicallydivided to fit and fetch into an internal memory according to one aspectof the present invention.

FIG. 16 is an exemplary flowchart of a method for dynamically dividingan output block in an output frame to fit and fetch into an internalmemory according to one aspect of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

While this invention is susceptible of embodiment in many differentforms, there is shown in the drawings and will herein be described indetailed preferred embodiment of the invention with the understandingthat the present disclosure is to be considered as an exemplification ofthe principles of the invention and is not intended to limit the broadaspect of the invention to the embodiment illustrated.

The numerous innovative teachings of the present application will bedescribed with particular reference to the presently preferredembodiment, wherein these innovative teachings are advantageouslyapplied to the particular problems of a geometric correction engine.However, it should be understood that this embodiment is only oneexample of the many advantageous uses of the innovative teachingsherein. In general, statements made in the specification of the presentapplication do not necessarily limit any of the various claimedinventions. Moreover, some statements may apply to some inventivefeatures but not to others.

While this invention has been particularly shown and described withreference to preferred embodiments, it will be understood by thoseskilled in the art that various changes in form and detail may be madetherein without departing from the spirit and scope of the invention.The inventors expect skilled artisans to employ such variations asappropriate, and the inventors intend the invention to be practicedotherwise than as specifically described herein. Accordingly, thisinvention includes all modifications and equivalents of the subjectmatter recited in the claims appended hereto as permitted by applicablelaw. Moreover, any combination of the above-described elements in allpossible variations thereof is encompassed by the invention unlessotherwise indicated herein or otherwise clearly contradicted by context.

It should be noted that the terms “output image” and “output frame” areinterchangeably used herein after to indicate an image displayed on ascreen similar to frame (0202).

It should be noted that the terms “input image” and “input frame” areinterchangeably used herein after to indicate an image displayed on ascreen similar to frame (0201).

It should be noted that the terms “output block size” and “size ofoutput block” are interchangeably used herein after to represent thesize of an output block in an output frame. The output block size isdefined by an output block width and an output block height. The terms“output block width” or the “width of the output block” areinterchangeably used herein after to denote number of pixels arranged ina line. Similarly, the terms “output block height” or the “height of theoutput block” are interchangeably used herein after to denote number oflines in the output block. For example, an output block size may bedefined as 16×20 to indicate an output block width of 16 and an outputblock height of 20. In other words, the number of pixels in the outputblock is 16 arranged in a row and the number of lines in the block is20.

It should be noted that the terms “region size” and “size of region” areinterchangeably used herein after to represent the size of a region inan output frame. The region size is defined by a region width and aregion height. The region may comprise one or more output blocks. Forexample region size may be defined as 64×40 to indicate a region widthof 64 and region height of 40. The region may be divided into equaloutput blocks with an output block size of 16×20. The output blocks size16×20 may be arranged in 4 columns and the number of rows in the columnis 2.

Histogram of Scaling Data with Scaling Thresholds (0500)

FIG. 5 depicts an exemplary histogram (0500) of scaling data of outputblocks of FIG. 2. After an image is initially captured, a distributionof magnification factor (ratio of fetched block size compared to outputblock size) may be plotted on the x-axis versus the number of blocks onthe y-axis. The histogram may be plotted by an offline utility or withsoftware running with HLOS or RTOS after the magnification data iscomputed and stored for each of the output blocks in the output image.It should be noted that the terms “scaling data”, “scaling factor”,“magnification data”, “magnification factor” are interchangeably usedherein after to indicate a ratio of size of input blocks required to befetched for a corresponding size of an output block. Scaling thresholdsmay be determined based on the histogram (0500). For example, withreference to histogram (0500), 3 scaling thresholds (0501, 0502, and0505) are computed or in some cases chosen. Scaling threshold (0501)indicates a scaling factor of approximately 40 and the number of blockswith a scaling factor 40 is approximately 5. Similarly, scalingthreshold (0505) indicates a scaling factor of approximately 5 and thenumber of blocks with a scaling factor 5 is more than 1200. Prior artmethods fixed the output block size based on the highest scaling factorin the output frame. In this case, the block size is based on thescaling factor of 40. The combination of a smaller block size and alarge scaling factor results in fetching undesired input block data foroutput blocks with smaller block size. For example, if the output blocksize if determined to be 16×20, an output block with a scaling factor 5(“block5”) and output block with a scaling factor 40 (“block40”) fetchthe same maximum amount of input block size corresponding to a scalingfactor of 40. In this particular example, the block5 would be fetchingthe same amount of input block data as block40 even though block5requires a substantially smaller amount of input data to be fetched, inthis case a factor of 8 times less. In an application with wide varyingmagnification data (like surround view) where selection of leastdimension block size leads to additional memory band-width, powerincrease and performance reduction by 50-100% based on size of localmemory and correction factors.

Spatial Plot of Scaling Data Across an Output Frame (0520)

According to a preferred exemplary embodiment, a spatial plot (0520) ofthe scaling data of the output blocks based on the scaling thresholds isplotted for the output frame. The thresholds identified in the histogram(0500) enables the output frame to be divided into regions based on thethresholds. As illustrated in the spatial plot (0520), the area (0504)represents area with minimum magnification factors and the area (0503)represents area with maximum magnification factors. According to apreferred exemplary embodiment, the output frame is divided into regionsbased on the scaling factors. According to a further exemplaryembodiment, the output block size is determined by the scaling factorwithin the region. It should be noted that the scaling factor within aregion may be substantially the same. In other instances the scalingfactor within a region may vary and within 200% of each other. In otherinstances the scaling factor within a region may vary and within 20-40%of each other. The more the number of thresholds selected from thehistogram the more the number of available block sizes and the moreflexibility in dividing the output frame into regions with identicalscaling data.

Exemplary Spatially Adaptive Slicing Embodiment (0600)

FIG. 6 (0600) is a depiction of an exemplary output frame divided intoregions and each of the regions divided into output blocks withdifferent output block sizes based on the scaling data. After scalingthresholds are identified as depicted in the histogram (0500) andspatially plotting the scaling data, the output frame may be dividedinto multiple regions. The process may include identifying and spatiallyslicing the output frame into multiple slicing regions while adapting tothe pattern of the transformation and the scaling data.

It should be noted that the output frame (0600) divided into 9 regionsmay not be construed as a limitation. The output frame may be dividedinto any number of regions as possible given the hardware limitations.The frame may be divided into 3 horizontal slices (0611, 0612, and 0613)corresponding to 3 heights RH1, RH2 and RH3, and frame may be dividedinto up to 3 vertical slices (0601, 0602, and 0603) corresponding to 3widths RW1, RW2 and RW3. A total of 9 regions (0621-0629) may be createdfor the frame. Each region may be programmed with independent outputblock size. The output frame may be programmed with bigger block sizefor region with less spatial variation (i.e. scaling factor) and smallerblock size for region with high spatial variation. This may improve theband width as well as performance as portion of the image with biggerblock size increases. For example, in surround view application, theblock size may vary from 16×20 to 112×64 for right camera and 32×8 to136×160 for a front camera. The output frame may be programmedindividually for input images from each of the cameras. According to anexemplary embodiment greater than 40% saving in band width may beachieved. According to a more preferred exemplary embodiment a greaterthan 80% saving in band width may be achieved. According to a mostpreferred exemplary embodiment a greater than 500% saving in band widthmay be achieved

Each of the regions may be programmed with a different block size asdepicted in the output frame (0600). Region (0624), which has themaximum scaling factor (0503), is divided into 16 blocks with a minimumblock size, while region (0623) which has the minimum scaling factor(0504) is divided into 2 blocks with a maximum block size. Similarly,regions (0622) and (0621) are divided into 4 blocks, regions (0623),(0626), and (0629) are divided into 2 blocks each, region (0625) isdivided into 6 blocks and region (0628) is divided into 9 blocks. Asclearly illustrated the output block is spatially sliced into regionswith varying block sizes. According to a preferred exemplary embodiment,the size of each of the regions in the output frame may be equal.According to another preferred exemplary embodiment, the size of each ofthe regions in the output frame may not be equal. The output frame maybe composed or constructed one region at a time in a raster scan mode.For example, region 0621 may be processed first, followed by regions0622, 0623, 0624, 0625, 0626, 0627, 0628, and 0629 in that order. Withina region, a raster scan may be followed to compose the region. Forexample, within region 0621, block 0631 is processed first followed byblocks 0641, 0651, and 0661 in that order. Similarly, within region0625, block 0635 is processed first followed by blocks 0645, 0655, 0665,0675, and 0685 in that order. Similarly, block 0633 is processedfollowed by block 0643. The processing of each of the block is furtherdescribed below in the method flow chart of FIG. 11. According to anexemplary embodiment, the coordinates of each of the region and each ofthe blocks, the size of each of the regions and each of the blockswithin the region are programmed as input parameters into a hardwareblock to process the output image as described below in FIG. 8.

When a frame is divided into sub-set of 3×3 regions, the followingcombinations of regions may be supported.

-   -   3 vertical slices RW1=x, RW2=y and RW3=z    -   2 vertical slices RW1=x, RW2=y and RW3=0, Last region width has        to be zero    -   1 vertical slice RW1=x, RW2=0 and RW3=0, Last two regions width        has to be zero    -   3 horizontal slices RH1=x, RH2=y and RH3=z    -   2 horizontal slices RH1=x, RH2=y and RH3=0 Last region height        has to be zero    -   1 horizontal slice RH1=x, RH2=0 and RH3=0 Last two regions        height has to be zero    -   A 2×2 region partitioning can be done by programming RW3=RH3=0.

Exemplary Geometric Image Processing System

FIG. 7 illustrates an exemplary image processing system (0700) accordingto one preferred embodiment. The system comprises a spatial slicingapparatus block (0703), a scalar block, a noise filter block, a sensorblock, a shared memory block (0704), a message manager (0702), a networkconfiguration block (0701), internal memory (0705), a DMA port (0706),and a hardware thread scheduler block.

The spatial adaptive slicing apparatus (0703) reads input frame frommaster port (0707) via DDR or on-chip and performs perspective transformas well as correction of distortion (including fisheye lenses). Theoutput of apparatus (0703) can be sent to external memory (DDR) or sentto other hardware blocks such as scalar block or noise filter block forfurther pre-processing via local shared memory (0705). Spatial adaptiveslicing apparatus (0703) may be targeted to operate at 120 frames persecond (FPS) @300 MHZ with 2M pixel frame. That performance requirementmay translate to BW of approximately 1.5 GBs. In order to meet thebandwidth requirement of 1.5 GBs, a read master interface may be limitedto maximum response latency of 200 cycles.

The scalar block reads data from shared memory (0704) and may generateup to 10 scaled outputs from 2 inputs with various scaling ratiosranging from 0.5 to 1. The output of scalar block to shared memory (SL2)can be further noise filtered using noise filter block or written toDDR.

The noise filter block reads data from memory (eg. DDR or on-chip) toshared memory (0704) and performs bilateral filtering to remove noise.The output of noise filter block can be sent to external memory (e.g.DDR) from shared memory (0704) or can be further re-sized using thescalar block.

The shared level 2 (0704) memory block may be used to exchange dataacross hardware blocks such as apparatus (0703), scalar block, and noisefilter block as well as to DMA Engine (UDMA).

A HTS (Hardware Thread Scheduler) block may be used for IPCcommunication among various hardware blocks such as apparatus (0703),scalar block, and noise filter block as well as to DMA Engine (UDMA).The message manager (0702) may be implemented as HTS.

The configuration manager (0701) may be used to program the hardwarewith typical network parameters. The system (0700) may further comprisetypical hardware inputs such as clock, reset, network, data, and debugsignals.

Exemplary Adaptive Slicing Apparatus Embodiment (0800)

As shown in FIG. 8, the adaptive slicing apparatus (0800) comprises acounter block (0802) configured to compute output coordinates (0803) ofthe output frame for a region based on a size of the output block of theregion; a transform block (0804) and back mapping block (0806)configured to generate input coordinates (0807) corresponding to each ofthe output coordinates; a bounding module (0808) configured to computeinput blocks corresponding to each of the input coordinates and maycomprise displacement data of each of each output block; a buffer module(0809) configured to fetch data corresponding to each of the inputblocks and store in an internal memory; and an interpolation module(0810) configured interpolate data received from the buffer module;wherein the interpolated data for each of the regions is stitched,composed, and displayed on a display module. An external memory block(0801) may store the input frame, the external memory is external toapparatus. The frame buffer or the external memory block (0801) isexternal to the apparatus (0800) and in most instances, an off-chipSDRAM or a DDR memory. The external memory blocks stores fetch datacorresponding to input blocks of an input frame such as frame (0201).According to a preferred exemplary embodiment the input frame iscaptured with an optical device; the optical device is selected from agroup consisting of: wide angle lens, fish eye lens or any kind ofautomotive camera lens.

Given the coordinates of the undistorted output image such as image(0600), the corresponding coordinates of the distorted input image maybe calculated by combining the output coordinates and the offsets froman offset table. Distorted pixels from the input frame are read from theframe buffer, and buffered for the bilinear interpolation. After theinterpolation, corrected image may be written back and stored to the SL2shared memory.

The counter block (0802) computes output coordinates (0803) of theoutput frame for a region based on a size of the output block of theregion. For example, for region (0621) in FIG. 6, the output coordinates(0803) of block 0631 may include the coordinates of the 4 corners of theblock 0631. The coordinates may be computed by counter block based on anoutput frame size that includes a frame width and a frame height.Coordinates of (0,0) (frame start co-ordinate programmable in theapparatus) may be assumed as the left corner of the frame and thecoordinates of each of the regions and each of the blocks in the regionmay be computed. The input parameters such as region width and regionheight, block size, and block width provided to the slicing apparatusmay in turn be input into the counter block. According to a preferredexemplary embodiment, input parameters to the apparatus comprise outputframe width and output frame height, region width and region height;output block width and output block height within a region. The counterblock may compute the output coordinates (0803) as (h_(u) v_(u)).

In order to carry out geometric correction efficiently in time and framebuffer traffic, the hardware processes the output frame in small outputblocks. Software running on HLOS or RTOS may configure appropriateparameters then initiate spatially adaptive slicing function by writingto a control register in the hardware. The hardware may store andmaintain a bank of registers used for storing control and datainformation. The hardware controls the sequencing through output blocks,DMA transfers, and computation to process an entire image autonomously.An interrupt, if enabled, is asserted at the completion of theprocessing and composition of the output image.

The hardware may also be stalled and controlled on a macro-block basisby an external controller. An intermediate interrupt may be provided bythe apparatus (0800) to facilitate the stalling of the hardware.Stalling the apparatus (0800) may be achieved by deasserting a writerequest enable on the output write port.

When a camera is viewing a scene from two different positions or whenmultiple cameras are viewing the scene from different positions, atransformation between the two viewing angles is needed to align theimages. Under specific conditions, the class of geometrictransformations known as “homography”, or planar-perspectivetransformation, will capture the geometric relationship between theimages accurately. Common applications of homography transforms are toalign (or stitch) multiple frames of the same scene to compute apanoramic output image. A second application is the alignment of planarsurfaces in the world. Finally, perspective transforms are also usefulin computing depth maps from a stereo image pair. By rectifying the twoviews, the search to compute disparity between the two views issimplified to a 1-D search problem. The homography is defined by a 3×3transformation matrix, as in

h _(aff) =a* _(h) u+b*v _(u) +c  (1)

v _(aff) =d*h _(u) +e*v _(u) +f  (2)

z=g*h _(u) +h*v _(u)+1  (3)

h _(p) =h _(aff) /z

v _(p) =v _(aff) /Z

The affine transform is a subset of the perspective transformation. Bysetting g=h=0, h_(p)=haff and v_(p)=vaff. The mapping from destinationcoordinate to the source coordinate is expressed as

h _(aff) =a*h _(u) +b*v _(u) +c

v _(aff) =d*h _(u) +e*v _(u) +f

Where h_(u) and v_(u) are horizontal and vertical coordinates of thesource coordinates and h_(eff) and v_(eff) are the horizontal andvertical coordinates of the destination coordinates, a, b, c, and farecorrelation and transform factors.

In image alignment applications, the homography matrices may be computedby locating corresponding points in the two frames and estimating thematrix parameters to transform the set of points in one frame onto thecorresponding points in the second frame. In the stereo rectificationapplication, the matrix is determined (pre-computed) at the calibrationstep and remains fixed.

In YCbCr mode, the offset table (0808) defines a (□x, □y) vector for aregular grid of output points. The grid can be fully sampled or downsampled. A fully sampled grid will define an offset vector for everyoutput pixel, defining where to fetch the input data to compute theoutput pixel. This is the most precise definition and can capturerapidly changing offset tables. The drawback is that it will require alarge amount of memory bandwidth as the geometric correction engine willbe reading offset values for every output pixel. Since most offsettables are not expected to change rapidly in a small spatial region, asubsampled offset table may be read. Offset tables can be subsampled bypowers of two in both horizontal and vertical directions and thesubsampling factor is set in a register. This mode conserves memorybandwidth by reducing the amount of data read to describe the offsetvectors, but requires more hardware to interpolate the missing offsetvectors.

x ₁ =x _(o) +Δx

y ₁ =y _(o) +Δt

The output coordinates (0803) (h_(u) v_(u)) from the counter block maybe input to a perspective transform block (0804) and aftertransformation using any of the equations (1,2,3) aforementioned thetransformed output coordinates (0805) (h_(p) v_(p)) may be input to amesh mapping block (0806). If back mapping is enabled in a mux block(0815), the coordinates (0807) (h_(d),v_(d)) calculated by the backmapping block is input to the buffer block (0809). If back mapping isnot enabled in a mux block (0815), the coordinates (0805) (h_(p),v_(p))calculated by the perspective transform block (0804) is input to thebuffer block (0809).

FIGS. 9 (0900) and 10 (1000) illustrate an exemplary mesh mapping andback mapping blocks according to a preferred embodiment. The perspectivetransform block (0804) does perspective warping on 4 corners of theoutput block to find out mesh block to fetch. The input coordinates(0805) (h_(p) v_(p)) to a mesh mapping block (0806) such as coordinate(0911) is subsampled. Rectangular bounding box is calculated such thatit contains mesh points referred by 4 corner pixels (sub-sampled)generated by perspective warp block. Bilinear interpolation may be doneto up-sample the mesh sub-sample to pixel co-ordinate precision. For themesh bilinear interpolation, one extra column on right and one extra atthe bottom may be fetched. For a given output block (0920), perspectivewrap coordinates such as 0911 for top left corner are calculated andbounding box (0930) is generated after sub sampling mesh points (0901,0902, 0903, 0904). After adding a buffer, a final output mesh block(0940) is generated.

Once mesh block fetch is completed, final input co-ordinates arecalculated by applying back mapping on previously calculated perspectivewarp corner pixel co-ordinates. Additional padding is applied on top ofthese back mapped corner co-ordinates based on the interpolation type.

It is possible that the bounding box calculated by hardware may notcover all the input data required to generate particular output block.In such cases, software may apply an additional PixelPad, the amount ofpadding in input block in all directions. For each output pixel in theoutput block with a size OBW×OBH, the input pixels required are indeedbounded by back mapping of the 4 corners plus/minus the padding. Moreprecisely, the input block may be determined by the following equations:

IBX_start=min(truncate(distortx(corner1)),truncate(distortx(corner3)))−Hw_Pad−PixelPad

IBX_end=max(truncate(distortx(corner2)),truncate(distortx(corner4)))+Hw_Pad+PixelPad

IBY_start=min(truncate(distorty(corner1)),truncate(distorty(corner2)))−Hw_Pad−PixelPad

IBY_end=maxn(truncate(distorty(corner3)),truncate(distorty(corner4)))+Hw_Pad+PixelPad

where corner1, corner2, corner3, and corner4 are upper-left,upper-right, lower-left, and lower-right corners of the OBW x OBH outputblock, and distortx(.), distorty(.) are X and Y coordinates of thecorners after perspective wrap and back mapping.

For a geometric distortion correction, PixelPad is zeroed out toaccommodate neighbor sets for all colors. Software may set the PixelPadsuch that information of the input blocks is not dropped. According to apreferred exemplary embodiment, the bounding module is furtherconfigured to add a buffer pixel pad to each of the input blocksincreasing the size of the input blocks.

OBH and OBW may be chosen as reasonably large for efficient operation ofadaptive slicing and geometric correction operation. According to apreferred exemplary embodiment, the region width ranges from 4 to 8094;and the region height ranges from 2 to 8094 According to a preferredexemplary embodiment the output block width ranges from 4 to 254; andthe output block height ranges from 2 to 254. OBW is constrained toensure efficient external memory write. Another constraint is that inputblock size, for each input block of the image, the allocated inputbuffer needs to accommodate without overflowing. If the parameters OBH,OBW are set too small, or PixelPad too large, performance may degradeand unintended and undesired external memory transfer may be happen.

An off-line utility program may be utilized to program and configure theapparatus (0800). Given offset table contents, processing parameters,and maximal input buffer size, the program computes an optimal set ofOBW, OBH, and PixelPad to minimize input bandwidth. Another utility maybe provided with a functional C model that computes the minimum PixelPadgiven a configuration and the processing block size with OBW (outputblock width) and OBH (output block height).

The hardware can be utilized to process a portion of the image, ratherthan the whole image. This allows an image to process through multiplesoftware/hardware interactions to correct only a portion of the image tosave time.

An intermediate interrupt, may also be provided on completion of eachmacroblock output write. This allows the geometric correction operationto be pipelined with other tasks. The apparatus (0800) output writestall after this event, waiting for a pulse on a start signal to beginwriting the next macro block.

As the coordinates (0807) (h_(d),v_(d)) calculated by the back mappingblock (0806) are not integer values in most cases, either bi-cubic orbilinear interpolation is applied to the distorted pixels with theinterpolation block (0810). Other interpolation techniques may also beapplied. According to a preferred exemplary embodiment, interpolationdata is interpolated with a bi-cubic or bilinear interpolation.Depending on register configuration, either bi-cubic or bilinearinterpolation is used to interpolate the output Y pixels. In the case ofbi-cubic interpolation, the distorted input pixel is interpolated fromthe 16 Y pixels in the 4×4 grid around the distorted input location, asshown in FIG. 6. Bi-cubic interpolation is used first along thehorizontal direction, then the vertical direction. FIG. 10 (1020)illustrates a Bicubic Interpolation of data (1021) in an output block(1022). The interpolated data may be further stored in a memory (0813)though an interface (0814). After the input coordinates corresponding tothe input blocks to be fetched are determined, the input blocks arefetched into an internal memory in a buffer (0809) through a bufferinterface (0811). According to a preferred exemplary embodiment, thesize of the internal memory ranges from 10 KB to 100 KB. In someinstances the size of the internal memory ranges from 1 KB to 100 KB. Inother instances size of the internal memory ranges from 10 KB to 50 KB.

FIG. 10 (1000) depicts an exemplary output coordinates to inputcoordinates after perspective mapping and back mapping. According to apreferred exemplary embodiment, the output coordinates comprise thecoordinates of the four corners of a block in the output frame/region.Individual coordinates for each of the corners may also be provided asthe output coordinates. According to a preferred exemplary embodimentthe input coordinates comprise the coordinates of the four corners of ablock in the input frame after perspective transform of the outputcoordinates and back mapping. Individual coordinates for each of thecorners may also be provided as the input coordinates. An output blockdefined by the corners 1001, 1002, 1003, and 1004 are mapped to inputblock corners 1011, 1012, 1013, and 1014, respectively. The final inputblock size is padded up with a size (1005) so that a larger input block(1010) is fetched and information is not lost in the final output image.According to a preferred exemplary embodiment, the interpolated data isfurther processed through modules configured to filter, reduce noise,scaled and formatted. According to another preferred exemplaryembodiment, the output frame is displayed on a display device; thedisplay device selected from a group consisting of: automotive display,LED monitor, television screen, and LCD monitor. According to apreferred exemplary embodiment, the output frame is processed in araster scan mode within each block in a region and within each region

Exemplary Geometric Correction of an Input Frame with a SpatiallyAdaptive Apparatus Method (1100).

As generally seen in the flow chart of FIG. 11 (1100), a preferredexemplary geometric correction method may be generally described interms of the following steps:

-   -   1) allocating a size for an output frame and dividing the output        frame into blocks; each of the blocks having a block size; the        block size having a width and a height; the width based on the        pixels arranged in a line; the height based on the number of        lines (1101);        -   As described in FIG. 2, the size of the output frame may be            allocated based on the frame to be displayed on a screen.            Software or a utility may determine, allocate, and configure            a hardware register with a frame size comprising a frame            width and a frame height. For example, the output frame for            a 4K display may be allotted to be 3840×2160. In another            example a HD display of 1920×1080 or 2 Mega pixel may be            allotted.    -   2) capturing the input image with an optical device (1102);        -   The captured image may be stored in an external memory with            the individual fetch blocks.    -   3) mapping back each of the blocks in the output frame to        correspond to blocks in the input image (1103);        -   Software or a utility may determine and identify fetch            blocks corresponding to each of the output blocks.    -   4) quantifying scaling data of each of the blocks in the output        frame (1104); Scaling thresholds may be determined based on the        histogram (0500). For example, with reference to histogram        (0500), 3 scaling thresholds (0501, 0502, and 0505 are computed        or in some cases chosen. Scaling threshold (0501) indicates a        scaling factor of approximately 40 and the number of blocks with        a scaling factor 40 is approximately 5. Similarly, scaling        threshold (0505) indicates a scaling factor of approximately 5        and the number of blocks with a scaling factor 5 is more than        1200.    -   5) generating a histogram from the scaling data of the output        frame (1105); a distribution of magnification factor (ratio of        fetched block size compared to output block size) may be plotted        on the x-axis versus the number of blocks. The histogram may be        plotted by an offline utility or with software running with HLOS        or RTOS after the magnification data is computed and stored for        each of the output blocks in the output image    -   6) identifying scaling thresholds from the histogram (1106);    -   7) plotting a spatial domain plot of the output frame with the        scaling data and the scaling thresholds (1107);        -   a spatial plot (0520) of the scaling data of the output            blocks based on the scaling thresholds may be plotted for            the output frame.    -   8) dividing the output frame into regions based on the spatial        domain plot and the scaling thresholds (1108);    -   9) computing and dividing each of the regions into output blocks        based on the scaling thresholds and the scaling data within the        region (1109); The output frame may be divided into any number        of regions as possible given the hardware limitations. For        example, the frame may be divided into up to 3 horizontal slices        (0611, 0612, and 0613) and 3 vertical slices (0601, 0602, and        0603) for a total of 9 regions (0621-0629). Each region may be        programmed with independent output block size. The output frame        may be programmed with bigger block size for region with less        spatial variation (i.e. scaling factor) and smaller block size        for region with high spatial variation. This may improve the        band width as well as performance as portion of the image with        bigger block size increases. For example, in surround view        application the block size may vary from 16×20 to 112×64 for        right camera and 32×8 to 136×160 for a front camera.    -   10) programming the spatially adaptive slicing apparatus with a        size for each of the regions and a size for each of the output        blocks in each of the regions (1110); and        -   For example an output block size may be defined as 16×20 to            indicate an output block width of 16 and an output block            height of 20. In other words, the number of pixels in the            output block is 16 arranged in a row and the number of lines            in the block is 20. The region size is defined by a region            width and a region height. The region may comprise one or            more output blocks. For example region size may be defined            as 64×40 to indicate a region width of 64 and region height            of 40. The region may be divided into equal output blocks            with an output block size of 16×20. The output blocks size            16×20 may be arranged in 4 columns and the number of rows in            the column is 2.    -   11) correcting the input frame geometrically for each of the        regions across the output frame, composing and displaying the        output frame on a graphical device (1111).

FIG. 12 illustrates an exemplary diagram of an output frame (1200) withredundant blocks identified in regions according to a preferredembodiment. The output frame may be divided into regions (1201, 1202,1203, 1204, 1205, 1206, 1207, 1208, and 1209). The regions may be equalor unequal in size. The regions may further comprise output blocks.According to a preferred exemplary embodiment, the output frame may bedivided with the method (1100). Input images are captured with multiplecameras and the output image is blended and stitched from the inputimages. An input image of a place or an object may be captured by acamera or any optical device with a particular view. Another input imagemay be captured by a camera or any optical device with a particular viewand with the camera mounted at a different position. Similarly, multipleinput images may be captured with different cameras mounted at differentplaces and capturing the same input image from a different view. Forexamples in FIG. 3, images 0301, 0302, 0303, and 0304 are captured withcameras mounted on the front, left, right, and rear of an automobilerespectively. The shapes of the input images may also be arbitrary.According to a preferred exemplary embodiment, the output image may beshaped as a rectangle irrespective of the arbitrary shapes of the inputimages. The multiple images captured from different viewpoints containredundant information. For example, the region (0314) captured by acentral camera is also captured in the region (0313) by a left cameraand captured in the region (0315) by a right camera. Similarly, in FIG.12, regions 1204, 1205, and 1206 contain redundant regions. Each of theidentified redundant regions may contain one or more valid blocks andone or more redundant blocks. The redundant regions are double processedfor stitching and processing the same image. There is considerable lossof performance due to cycles spent in fetching and processing theredundant blocks. According to a preferred exemplary embodiment, anexemplary apparatus identifies and skips the processing of the redundantblocks.

Due to redundancy, some regions of the frame may not be used forstitching of final frame. By programming a register to disable theprocessing of the region, hardware skips the processing of particularregion that is identified as redundant. A frame done signal may be senton last pixel of last valid region. As an example shown in FIG. 12,region 1204, region 1205 and region 1206 are skipped for processing. Aframe done signal may be passed on last pixel of region 1207. Orderingand numbering of the regions will remain the same even in the case ofskipping some regions.

A portion (1301) of an output image (1303) may be processed whileskipping some regions, the following parameters may be programmed intoan apparatus such as apparatus (0800).

-   -   InitX X coordinate of upper-left corner of output frame    -   InitY Y coordinate of upper-left corner of output frame    -   FrameW Width of output compute frame    -   FrameH Height of output compute frame    -   FrameBase SDRAM address of upper-left corner of output frame (Y        and Cb/Cr base may be needed in case of YCbCr420).    -   FrameOfst SDRAM frame width (in bytes)        In some embodiments FrameW may be a multiple of OBW, FrameH may        a multiple of OBH, and FrameBase a multiple of 64 (byte        address).

FIG. 13 (1300) shows an exemplary preferred embodiment of multiple-passprocessing with middle-of-the-image being skipped in 8-bit format. Theapparatus (0800) does not copy skipped blocks from the input frame tothe output frame and software or a utility program may set up andinitiate the memory copy of the valid regions. The starting address ofinput frame, corresponding to input coordinate (0, 0), may be specifiedas a register as well. The apparatus (0800) may clip the input block toinput frame size if any of the input block falls outside input frame.The apparatus (0800) may function as an arbitrary shaped apparatus foruse in method (1400) with the output coordinates (0803) (h_(u) v_(u))from the counter block disabled for the regions that are skipped.Similarly, the apparatus (0800) may function as an dynamic partitionapparatus for use in method (1600) such that the coordinatescorresponding to the blocks after division is input to the counter blockprogrammed into the counter block and the output coordinates (0803)(h_(u) v_(u)) generated from the counter block.

Exemplary Geometric Correction of an Input Frame Method (1400).

As generally seen in the flow chart of FIG. 14 (1400), a preferredexemplary method for geometric correction of an input frame with anarbitrary shaped apparatus (0800) to generate an output frame may begenerally described in terms of the following steps:

-   -   1. capturing an arbitrary shaped input image with a plurality of        optical devices (1401);    -   2. processing each of the images captured with the plurality of        optical devices (1402);    -   3. identifying redundant blocks and valid blocks in each of the        regions (1403);    -   4. allocating an output frame with an output frame size and        dividing the output frame into regions shaped as a rectangle        (1404); Regions are decided after finding unused blocks for best        mapping of unused blocks to skipped regions.    -   5. programming the apparatus and disabling processing for each        of the redundant regions (1405); and    -   6. composing the output frame for each of the regions containing        the valid blocks(1406).

FIG. 15 (1500) illustrates an output block in an output frame thatdynamically divided to fit and fetch into an internal memory accordingto a preferred exemplary embodiment. An input buffer corresponding tooutput block (1501) may be fetched into an internal memory in a framebuffer from an external memory. According to an exemplary embodiment, ifa size of the input block (1502) corresponding to the output block(1501) fits into an internal memory (1512), then the input block (1502)is fetched into the internal memory. In this case, a dynamic division ofthe input block may not be required. For example, if the size of theinput block is 640×20 and the size of the internal memory is 12800Bytes, then the input block is completely fetched in one fetchtransaction or cycle. According to another preferred exemplaryembodiment, if the size of the input block corresponding to the outputblock (1501) is greater than the size of the internal memory, the inputblock is computed by dividing the output block into sub blocks. If thesize of each sub input block is less than the size of the internalmemory, the sub blocks may be each fetched individually or together intothe internal memory. Likewise, if the size of any sub input block isgreater than the size of the internal memory, and the sub output blocksare further divided until the size of the sub input blocks aftercomputation and division are less than the size of the internal memoryso that each sub input block after division is fetched into the internalmemory. For example, if the size of the input block (1502) is 640×20 andthe size of the internal memory is 6400 bytes, then the output block maybe dynamically divided into two sub blocks (1503 and 1513) so that thesize of the sub blocks are 640×10 each which fit into the internalmemory. If the size of the internal memory is 3200 bytes, then thedivided sub blocks are further dynamically divided into sub blocks(1514, 1524, 1534, and 1544). The division of the sub blocks may behorizontal or vertical. In other words the height of the input blocksmay be divided in a horizontal division and the width of the input blockmay be divided in a vertical division. In other examples the internalmemory may be fixed and input block size may varied from block to blockwith some input blocks bigger and some are smaller in a given frame.

As generally seen in the flow chart of FIG. 16 (1600), a preferredexemplary method for geometric correction of an input frame with andynamic partition apparatus (0800) to generate an output frame may begenerally described in terms of the following steps:

-   -   1. allocating an output frame with an output frame size and        dividing the output frame into output blocks; each of the output        blocks having an output block size; the output block size having        a width and a height; the width based on the pixels arranged in        a line; the height based on the number of lines (1601);    -   2. capturing an input image with an optical device (1602);    -   3. mapping back each of the output blocks in the output frame to        input blocks in the input image (1603);    -   4. computing a size of the input blocks in the input image        corresponding to each output blocks (1604);        -   Input block computation may happens in the following            sequence of processing. For each output block following            steps are done in sequence.        -   1. Calculate input block.        -   2. Fetch input block (assuming the size fits in internal            memory).        -   3. Processing of the pixels and write out.        -   The aforementioned steps (1-3) are performed in sequence of            each output block    -   5. for each of the output blocks, checking for size of the input        blocks with the size of the internal memory (1605);    -   6. if the size of the input blocks is less than the size of the        internal memory, programming an apparatus with input parameters,        fetching the input blocks into an internal memory and processing        the next output block in step (1605) until all the output blocks        are processed and proceeding to step (1609); if not, proceeding        to step (1605) (1606);        -   For example, if the size of the input block is 640×20 and            the size of the internal memory is 12800 bytes, then the            input block is completely fetched in one fetch transaction            or cycle or else step (1607) may be executed.    -   7. dividing the output block equally into sub blocks and        checking if the size of the sub block is less than the size of        the internal memory; if so, programming an apparatus with input        parameters, fetching the input blocks into an internal memory,        processing each of the sub blocks and processing the next output        block in step (1605) until all the output blocks are processed        and proceeding to step (1609); if not proceeding to step (1608)        (1607);    -   For example, if the size of the input block is 640×20 and the        size of the internal memory is 6400 bytes, then the input block        is divided into sub blocks. The size of the sub block is less        than or equal to the size of the internal memory, the sub blocks        are each fetched into the internal memory and processed        sequentially to generate an output block.    -   8. dividing further until the size of divided sub blocks is less        than the size of the internal memory, programming an apparatus        with input parameters, fetching the input blocks into an        internal memory, processing each of the divided sub blocks        sequentially and processing the next output block in step (1605)        until all the output blocks are processed (1608); and    -   9. composing the output frame for each of the blocks in the        output frame (1609).

System Summary

The present system may be broadly generalized as an apparatuscomprising:

-   -   (a) a counter block configured to compute output coordinates of        the output frame for a region based on a size of the output        block of the region;    -   (b) a transform and back mapping block configured to generate        input coordinates corresponding to each of the output        coordinates;    -   (c) a bounding module configured to compute input blocks        corresponding to each of the input coordinates;    -   (d) a buffer module configured to fetch data corresponding to        each of the input blocks and store in an internal memory; and    -   (e) an interpolation module configured interpolate data received        from the buffer module; wherein    -   the interpolated data for each of the regions is stitched,        composed, and displayed on a display module.

This general system summary may be augmented by the various elementsdescribed herein to produce a wide variety of invention embodimentsconsistent with this overall design description.

Method Summary

The present method may be broadly generalized as a method forgeometrically correcting an input image with a spatially adaptiveslicing apparatus and generating an output frame wherein:

the method comprising the steps of:

-   -   a) allocating a size for an output frame and dividing the output        frame into blocks; each of the blocks having a block size; the        block size having a width and a height, the width based on the        pixels arranged in a line; the height based on the number of        lines;    -   b) capturing the input image with an optical device;    -   c) mapping back each of the blocks in the output frame to        correspond to blocks in the input image;    -   d) quantifying scaling data of each of the blocks in the output        frame;    -   e) generating a histogram from the scaling data of the output        frame;    -   f) identifying scaling thresholds from the histogram;    -   g) plotting a spatial domain plot of the output frame with the        scaling data and the scaling thresholds;    -   h) dividing the output frame into regions based on the spatial        domain plot and the scaling thresholds;    -   i) computing and dividing each of the regions into output blocks        based on the scaling thresholds and the scaling data within the        region;    -   j) programming the spatially adaptive slicing apparatus with a        size for each of the regions and a size for each of the output        blocks in each of the regions; and    -   k) correcting the input frame geometrically for each of the        regions across the output frame, composing and displaying the        output frame on a graphical device.

This general method may be modified heavily depending on a number offactors, with rearrangement and/or addition/deletion of stepsanticipated by the scope of the present invention. Integration of thisand other preferred exemplary embodiment methods in conjunction with avariety of preferred exemplary embodiment systems described herein isanticipated by the overall scope of the present invention.

System/Method Variations

Embodiments of the present invention anticipates wide variety ofvariations in the basic theme of construction. The examples presentedpreviously do not represent the entire scope of possible usages. Theyare meant to cite a few of the almost limitless possibilities.

This basic system, method, and product-by-process may be augmented witha variety of ancillary embodiments, including but not limited to:

-   -   An embodiment wherein the optical device is selected from a        group consisting of: wide angle lens, fish eye lens, and        automotive camera lens.    -   An embodiment wherein the step of capturing the input image        further comprises storing the input image in an external memory.    -   An embodiment wherein the step of mapping back further comprises        generating input coordinates corresponding to each of the blocks        in the output frame; the input coordinates further comprising        pixel information in the input frame.    -   An embodiment wherein the size of the output frame configured        with an output frame height and a output frame width; the output        frame height ranges from 8 to 8094; and the output frame width        ranges from 8 to 8094;    -   An embodiment wherein the number of pixels in the line in step        ranges from 8 to 8094;    -   An embodiment wherein the number of lines in the line in step        ranges from 8 to 8094;    -   An embodiment wherein the step of mapping back further comprises        identifying output corners for each of the blocks; performing a        perspective transform on each of the corners; finding the input        corners corresponding to the output corners using back mapping        after the perspective transformation.    -   An embodiment wherein the step of quantifying scaling data        further comprises determining the number of input blocks        required to be fetched for each of the blocks in the output        frame.    -   An embodiment wherein the step of identifying scaling thresholds        is further determined by grouping output blocks with similar        scaling data.    -   An embodiment wherein the step of dividing the output frame into        regions further comprises grouping output blocks with identical        scaling data such that the size of each of the region is        maximized.    -   An embodiment wherein the step of programming the spatially        adaptive slicing apparatus further comprises writing a size of        each of the regions and a size of each of the blocks into a        register in a register bank; the register bank maintained in an        internal memory of the apparatus.    -   An embodiment wherein the size of each of the regions is        determined by a region width and a region height; the region        width ranges from 8 to 8094; and the region height ranges from 8        to 8094.    -   An embodiment wherein the size of each of the output blocks in        each of the regions is determined by an output block width and a        output block height; the output block width ranges from 4 to        254; and the output block height ranges from 2 to 254.    -   An embodiment wherein the step of correcting the output frame        further comprises correcting each of blocks in a region in a        raster scan mode and moving to next region in a raster scan        mode.    -   An embodiment wherein the step of displaying the output frame        further comprises fetching the input blocks for each of the        output blocks into an internal memory without overflowing the        internal memory.

One skilled in the art will recognize that other embodiments arepossible based on combinations of elements taught within the aboveinvention description.

We claim:
 1. A method for geometrically correcting an input image with aspatially adaptive slicing apparatus and generating an output frame,said method comprising the steps of: a) allocating a size for an outputframe and dividing the output frame into blocks; each of the blockshaving a block size; the block size having a width and a height; thewidth based on the pixels arranged in a line; the height based on thenumber of lines; b) capturing the input image with an optical device; c)mapping back each of the blocks in the output frame to correspond toblocks in the input image; d) quantifying scaling data of each of theblocks in the output frame; e) generating a histogram from the scalingdata of the output frame; f) identifying scaling thresholds from thehistogram; g) plotting a spatial domain plot of the output frame withthe scaling data and the scaling thresholds; h) dividing the outputframe into regions based on the spatial domain plot and the scalingthresholds; i) computing and dividing each of the regions into outputblocks based on the scaling thresholds and the scaling data within theregion; j) programming the spatially adaptive slicing apparatus with asize for each of the regions and a size for each of the output blocks ineach of the regions; and k) correcting the input frame geometrically foreach of the regions across the output frame, composing and displayingthe output frame on a graphical device.
 2. The method of claim 1 whereinthe optical device is selected from a group consisting of: wide anglelens, fish eye lens, and automotive camera lens.
 3. The method of claim1 wherein the step (b) of capturing the input image further comprisesstoring the input image in an external memory.
 4. The method of claim 1wherein the step (c) of mapping back further comprises generating inputcoordinates corresponding to each of the blocks in the output frame; theinput coordinates further comprising pixel information in the inputframe.
 5. The method of claim 1 wherein the size of the output frameconfigured with an output frame height and a output frame width; theoutput frame height ranges from 8 to 8094; and the output frame widthranges from 8 to 8094;
 6. The method of claim 1 wherein the number ofpixels in the line in step (a) ranges from 8 to 8094;
 7. The method ofclaim 1 wherein the number of lines in the frame in step (a) ranges from8 to 8094;
 8. The method of claim 1 wherein the step of mapping backfurther comprises identifying output corners for each of the blocks;performing a perspective transform on each of the corners; with backmapping finding the input corners corresponding to the output cornersafter the perspective transformation.
 9. The method of claim 1 whereinthe step (d) of quantifying scaling data further comprises determiningthe number of sub input blocks required to be fetched for each of theblocks in the output frame.
 10. The method of claim 1 wherein the step(f) of identifying scaling thresholds is further determined by groupingoutput blocks with similar scaling data.
 11. The method of claim 1wherein the step (h) of dividing the output frame into regions furthercomprises grouping output blocks with identical scaling data such thatthe size of each of the region is maximized.
 12. The method of claim 1wherein the step (h) of programming the spatially adaptive slicingapparatus further comprises writing a size of each of the regions and asize of each of the blocks into a register in a register bank; theregister bank maintained in an internal memory of the apparatus.
 13. Themethod of claim 1 wherein the size of each of the regions is determinedby a region width and a region height; the region width ranges from 4 to8094; and the region height ranges from 2 to
 8094. 14. The method ofclaim 1 wherein the size of each of the output blocks in each of theregions is determined by an output block width and a output blockheight; the output block width ranges from 4 to 254; and the outputblock height ranges from 2 to
 254. 15. The method of claim 1 wherein thestep (k) of correcting the output frame further comprises correctingeach of blocks in a region in a raster scan mode and moving to nextregion in a raster scan mode.
 16. The method of claim 1 wherein the step(k) of displaying the output frame further comprises fetching the inputblocks for each of the output blocks into an internal memory withoutoverflowing the internal memory.
 17. An apparatus for geometricallycorrecting an input frame and generating an output frame, said apparatuscomprising: a) a counter block configured to compute output coordinatesof the output frame for a region based on a size of the output block ofthe region; b) a transform and back mapping block configured to generateinput coordinates corresponding to each of the output coordinates; c) abounding module configured to compute input blocks corresponding to eachof the input coordinates; d) a buffer module configured to fetch datacorresponding to each of the input blocks and store in an internalmemory; and e) an interpolation module configured interpolate datareceived from the buffer module; wherein the interpolated data for eachof the regions is stitched, composed, and displayed on a display module.18. The apparatus of claim 17 wherein an external memory block isconfigured to store the input frame; said external memory external tothe apparatus.
 19. The apparatus of claim 17 wherein the input frame iscaptured with an optical device; the optical device is selected from agroup consisting of: wide angle lens, fish eye lens, and automotivecamera lens.
 20. The apparatus of claim 17 wherein the output frame isdisplayed on a display device; the display device selected from a groupconsisting of: automotive display, LED monitor, television screen, andLCD monitor.
 21. The apparatus of claim 17 wherein the outputcoordinates comprise the four corners of a block in the output frame.22. The apparatus of claim 17 wherein the input coordinates comprise thefour corners of a block in the input frame after perspective transformof the output coordinates and back mapping.
 23. The apparatus of claim17 wherein input parameters to the apparatus comprise output frame widthand output frame height, region width and region height; output blockwidth and output block height within a region.
 24. The apparatus ofclaim 17 wherein the region width ranges from 4 to 8094; and the regionheight ranges from 2 to
 8094. 25. The apparatus of claim 17 wherein theoutput block width ranges from 4 to 254; and the output block heightranges from 2 to
 254. 26. The apparatus of claim 17 wherein the boundingmodule is further configured to add a buffer pixel pad to each of theinput blocks increasing the size of the input blocks.
 27. The apparatusof claim 17 wherein the size of the internal memory ranges from 10 KB to50 KB.
 28. The apparatus of claim 17 wherein interpolation data isinterpolated with a bi-cubic or bilinear interpolation.
 29. Theapparatus of claim 17 wherein the output frame is processed in a rasterscan mode within each block in a region and within each region.
 30. Theapparatus of claim 17 wherein the interpolated data is further processedthrough modules configured to filter, reduce noise, scaled andformatted.